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ABSTRACT 

In  this  paper  we  presenfc^several  results  for  three  different  canonical 
forms  of  linear  prediction  on  a  plane.  These  filters  have  causal,  semicausal 
and  noncausal  prediction  geometries.  Starting  from  their  properties -we  consider 
the  problem  of  realization  of  these  filters  from  a  given  power  spectral  density 
function  (SDF).  Since  it  is  not  possible  in  general  to  obtain  rational  spectral 
factors  of  a  two  dimensional  SDF,  we:  propose  algorithms  for  obtaining  rational 
approximations  which  are  stable  and  converge  to  their  limit  (irrational)  factors 
as  the  order  of  approximation  is  increased.  It  is  also  shown  that  the  normal 
equations  associated  with  the  minimum  variance  two-dimensional  prediction 
filters  give  a  useful  algorithm  for  obtaining  rational  approximations  which 
are  stable  and  converge  to  their  unique  limit  filters.  This  result  allows 
design  of  finite  order,  stable  filters  by  solving  a  finite  number, of  equations 
while  realizing  the  given  SDF  arbitrarily  closely.  ^ 

\ 
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I.  INTRODUCTION 

Many  digital  signal  processing  algorithms  are  designed  for  the  processing 
of  random  data.  For  instance,  a  digital  image  may  be  considered  to  be  a  sample 
function  of  a  discrete,  wide  sense  stationary  random  field,  which  is  described 
in  terms  of  its  covariance  function  or  equivalently  the  spectral  density  function 
(SOF).  Although  algorithms  such  as  Wiener  filtering  [1],  transform  coding  of 
images  [2],  etc.  can  be  designed  once  the  covariance  function  is  given,  it  is 
sometimes  useful  to  obtain  a  linear  difference  equation  representation  of  the 
random  field.  The  model,  if  sufficiently  accurate,  should  realize  the  SDF  of 
the  random  field  closely.  Such  models  are  useful  in  image  coding,  recursive 
filtering,  image  synthesis,  spectral  estimation,  etc. 

These  models  can  be  realized  by  spectral  factorization.  In  the  one  dimen¬ 
sional  ( 1 -D )  case,  this  involves  factoring  the  SDF  of  the  random  process  into 
causal  (recurs  ve)  and  anticausal  factors.  The  general  techniques  to  do  this 
include  the  Wiener-Doob  method  [3]  and  the  linear  prediction  method  [4].  The 
principle  of  the  Wiener-Doob  method  is  to  map  the  poles  and  zeros  of  the  SDF 
into  singularities  of  the  logarithm  of  the  SDF.  This  allows  easy  decomposition 
of  singularities  into  those  that  are  within  the  unit  circle,  and  into  those  that 
are  outside,  leading  to  minimum  phase  and  maximum  phase  factors,  respectively. 

The  linear  prediction  method  fits  successively  higher  order  autoregressive  models 
to  the  given  covariances  by  solving  a  finite  set  of  Toeplitz  equations.  Under 
some  mild  conditions  on  the  given  SDF,  the  SDFs  realized  by  the  models  can  be 
shown  to  converge  uniformly  to  the  given  SDF. 

In  two  dimensions  (2-D),  spectral  factorization  is  complicated  by  the  lack 
of  a  fundamental  theorem  of  algebra.  Whittle  [5]  recognized  this  difficulty, 
and  indicated  a  Wiener-Doob  like  technique  for  2-D  spectral  factorization.  This 
fact  seems  to  have  been  ignored  until  it  was  rediscovered  by  Ekstrom  and  Woods  [6]. 
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They  provided  an  extension  of  the  1-D  Wiener-Doob  principle  to  2-D  to  obtain 
recursive  filter  designs  from  given  frequency  magnitude  specifications.  The 
resulting  infinite  order  causal  factors  were  approximated  to  finite  order  by 
using  suitable  truncation  and  windowing. 

Subsequently,  Marzetta  [7,8]  approached  the  spectral  factorization  problem 
by  extending  the  results  of  1-D  prediction  theory  to  2-D.  The  recursive  factors 
obtained  here  are  of  infinite  order  in  at  least  one  of  the  dimensions.  Also, 
the  exact  solution  can  only  be  obtained  by  solving  an  infinite  set  of  linear 
equations.  A  2-D  analog  of  Levinson's  algorithm  was  devised  to  obtain  the  theore¬ 
tically  guaranteed  solution.  More  importantly,  it  was  shown  that  a  one-to-one 
relationship  exists  between  the  reflection  coefficients  obtained  in  the  Levinson 
algorithm,  the  associated  prediction  error  filters,  and  the  covariances  of  the 
random  field  used  in  the  algorithm.  Also,  it  was  proven  that  reflection  coeffi¬ 
cients  given  on  a  finite  support  yield  finite  support  causal  factors.  These  facts 
were  used  to  design  an  approximate  spectral  factorization  algorithm  where  the 
reflection  coefficients  are  sequentially  chosen  on  a  finite  lattice  to  minimize 
a  prediction  error  functional. 

In  [9],  Jain  studied  a  class  of  hyperbolic,  parabolic,  and  elliptic  partial 
differential  equations,  and  from  their  finite  difference  qpproximations  developed 
finite  order  causal,  semicausal,  and  noncausal  representation  for  2-D  random 
fields.  In  [10],  higher  order  models  were  considered,  and  it  was  shown  that  the 
coefficients  could  be  obtained  by  solving  a  finite  set  of  block  Toeplitz  equations. 
Though  a  unique  solution  may  be  found  for  the  model  parameters,  there  is  no  one- 
to-one  correspondence  between  the  coefficients  and  the  covariances  used  in  model 
realization.  Also,  model  stability  is  not  guaranteed.  The  advantage,  however, 
is  that  only  a  finite  Toeplitz  set  of  equations  need  be  solved. 

An  alternate  method  of  obtaining  models  by  factorizing  the  SDF,  S(z^  ,z2)  is 
shown  in  [7,8]  for  the  causal  case  and  In  [10]  for  both,  causal  and  semicausal 
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cases.  Here,  a  set  of  normal  equations,  parametric  in  z-j  must  be  sequentially 
solved  for  larger  and  larger  orders,  resulting  in  factors  which  are  irrational 
in  z-j .  Also  in  [10],  the  Wiener-Doob  principle  has  been  extended  to  obtain 
semicausal  representations. 

In  this  paper  we  present  several  results  concerning  realization  of  noncausal , 
semicausal  and  causal  models  by  linear  prediction  principles.  For  each  class  of 
models  we  present  two  types  of  algorithms,  A1  and  A2.  In  algorithm  A1  we  start 
with  the  given  power  spectrum  density  function  (SDF)  to  obtain  infinite  order 
(irrational)  realizations.  A  rationalization  procedure  is  incorporated  into  the 
algorithms  to  provide  stable  models.  These  algorithms  will  theoretically  require, 
as  in  all  prior  methods,  solution  of  an  infinite  number  of  equations.  A  Levinson 
type  algorithm  presented  for  semicausal  models  turns  out  to  be  an  extension 
of  a  similar  algorithm  [7,8]  for  causal  models.  The  results  for  noncausal  models 
do  not  require  spectral  factorization. 

The  algorithms  A2  yield  approximate  rational  realizations  of  2-D  SDFs  which 
are  obtained  by  solving  finite  number  of  equations.  Our  results  give  asymptotic 
behavior  of  properties  such  as  spectral  match,  stability,  covariance  match,  etc., 
of  these  models  as  their  order  is  increased  to  infinity.  An  important  consequence 
is  that  one  obtains  a  practical  procedure  for  designing  stable  causal,  semicausal 
and  noncausal  models  which  remain  finite  in  order  while  matching  the  given 
spectra  arbitrarily  closely. 

In  Section  II,  we  provide  the  necessary  definitions  and  properties  to  aid 
in  the  understanding  of  the  subsequent  sections.  One  dimensional  noncausal  models 
are  analyzed  in  Section  III,  since  the  considerations  involved  there  lead  to  insight 
into  2-D  model  bahaviour.  In  Section  IV,  we  present  results  on  realization  of 
2-D  noncausal  models,  by  solving  finite  order  block  Toeplitz  equations.  In  Sections 
V  and  VI,  results  for  semicausal  and  2-D  causal  models  are  presented.  In  Section 
VII,  we  give  examples  verifying  the  theoretical  results  of  the  earlier  sections. 
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II.  NOTATION,  DEFINITION  AND  PROPERTIES 

2.1  Notation 

a)  Two  dimensional  (2-0)  sequences  are  denoted  by  u(i,j),  e(i,j),  etc; 
where  (i,j)  are  integers  defined  on  a  regular  2-D  lattice. 

b)  Matrices  are  denoted  by  upper  case  letters,  e.g.  U,  A,  R,  etc.  For 
example, 

U  -  {u(i, j)  ;  1  1  i  1  N,  1  <  j  £  M} 

is  a  N  x  M  matrix  of  elements  u(i,j).  The  jth  column  of  U  is  written 
as  u.  and  the  (i,j)th  element  of  U  is  written  as  u(i,j). 

J 

c)  The  transpose  of  U  is  denoted  as  UT.  The  complex  conjugate  of  U  is 
denoted  as  U*. 

d)  The  z-transform  of  a  sequence  u(i,j)  is  denoted  as  UCz-j.z^),  and 
U(u>j,u>2)  denotes  the  z-transform  evaluated  on  the  unit  circles, 

lz] I  =  1 z2 1  a  ^ • 

e)  The  maximum  lower  bound  of  a  function  f(x)  on  a  domain  X  is  denoted 
as  inf Cf (x) ] ,  and  its  minimum  upper  bound  is  denoted  sup[f(x)]. 

2.2  Definitions  and  Properties 

1)  A  discrete  random  field  {u(i,j)>  is  a  2-D  sequence  of  random  variables 
u(i,j).  If  the  random  variables  are  real  and  jointly  Gaussian,  then 
it  is  called  a  real,  Gaussian  random  field. 

2)  The  random  field  (u(i,j)}  is  called  wide  sense  stationary  (or  just 
stationary)  if  its  mean  is  constant,  and  its  covariance  is  independent 
of  spatial  translation,  i.e. 

E[u(i,j)]  =  u(i,j)  *  u 

cov[u(1 ,j)u(m,n)]  ^  E[u(i,j)  -  y)(u(m,n)  -  u)]  *  ru(i-m,j-n) 

When  there  is  no  v  — ir  o *  infusion,  the  subscript  u  will  be  dropped 
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from  (*,*).  Henceforth,  we  will  consider  real,  zero  mean,  stationary 
random  fields. 

3)  The  random  field  (u ( i , j ) >  is  called  white  if  the  random  variables 
u(i,j)  are  mutually  uncorrelated,  i.e. 

ru(k,A)  =  a2S(k,Jt) 

2 

where  a  is  the  variance  of  the  field  and  6(k,£)  is  the  Kronecker 
delta  function. 

4)  The  spectral  density  function  (SDF) ,  S^.Zg)  of  {u(i,j)l  is  called 

positive  analytic  (PA)  [8]  if  S(o»1,w2)>0  and  if  S(z,,z2)  is  analytic  in 
a  neighborhood  of  { z-j  [  *  ~  1-  Let  this  neighborhood  be 

r  =  (y1<|z1 |<1/y1 ,  y2<|z2|<1/y2;  0<y1,y2<1}  (1) 

Then, 

a)  S(z-|,z2)  has  a  unique  Laurent  expansion  [11]  given  by 

S(z,,zJ  =  E  ?  r(k,t)z,'kz:Jl  ,  z,,z,  e  r  (2) 

*  k=-°°  1  1  L 


b)  Since  S(w-|  ,ui2)  >  0,  by  continuity,  there  exists  another 
neighborhood 

a  =  {cx1<|z1  l<l/ot1 ,  a2<|z2|<l/a2;  O^.a^l)  (3) 

where  S  ^(z^,z2)  is  PA,  and  has  the  Laurent  expansion 

s_1(z,,z2)  =  ?  ?  r'(k,st)z:kz;£  ,  z,  ,z„  e  a  (4) 


c)  Convergence  of  the  series  in  (2)  and  (4)  implies  that  the 
sequences  r(k,t)  and  r‘(k,£)  are  exponentially  bounded: 


|r(k,£)|  <  Ay^y^1 
*  ^ 

|r"(k,t) |  <  Bajkla^' 


m,n) 


(5) 


for  some  positive  constants  A,  B. 
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d)  The  sequence  of  functions  { r^(z-j ) >  defined  as 


rjl(z1)  =  ?  r(k,£)Zi 
k=-°° 


-k 


1 

2tt  j 


| z2 1 =1  4  ls(zl ,Z2)dz2 


(6) 


i)  are  analytic  in  the  neighborhood  {y,<|z.j  | <1  /yi > 
ii)  form  a  positive  definite  sequence  on  |z-||  =1,  i.e;  for 
any  x^(z^ )  not  identically  zero, 

xk(zl)rk-H(zl)x*(zl>  >  0  .  Iz,l  -  ! 


This  means  that  the  Toeplitz  matrix 


is  Hermitian  positive  definite  on  |z-||=  1. 
e)  The  sequence  {r(k,£.)}  is  also  positive  definite. 

The  following  definitions  and  properties  associated  with  linear  predic¬ 
tion,  which  are  discussed  in  greater  detail  in  [10],  will  be  needed  here. 

5)  Let  u(i,j)  denote  a  linear  prediction  (LP)  estimate  of  the  random 
variable  u(i  ,j) .  Then 

u(i,j)  =  Z  „  a(m,n)u(i-m,j-n)  (8) 

(m,n)dS 

A 

The  a(m,n)  are  called  the  predictor  coefficients  and  S,  a  subset  of 
the  2-D  lattice  is  called  the  prediction  region. 
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The  geometry  of  S  depends  on  the  type  of  prediction  estimate 
considered,  viz;  causal,  semi-causal  or  non-causal .  With  a  hypo¬ 
thetical  scanning  mechanism  that  scans  sequentially  from  top  to 
bottom  and  left  to  right,  the  three  prediction  regions  may  be  defined 


as  follows: 

Causal  Prediction: 

S-j  =  (m,n):{n>1  ,Vm}  U  {n=0,m>l}  (9) 

Semi -causal  Prediction: 

S2  =  (m,n):{n>l ,Vm}  U  {n=0,Vn*0}  (10) 

Non-causal  Prediction: 

S3  =  (m,n):{V(m,n)  t  (0,0)}  (11) 

Also,  we  define 

Si  =  Si  U  (0,0)  ,  i=l,2,3  (12) 

The  above  prediction  regions  are  depicted  in  Fig.  1.  in  practice. 


only  a  finite  number  of  nearest  neighbor  samples  from  prediction  windows 

/V  A 

M.  Cl  can  be  used  in  the  prediction.  Prediction  coefficients  (a(m,n)} 
defined  on  are  said  to  have  continuous  support  whereas  if  defined 
only  on  the  W.  they  have  discontinuous  support. 

The  prediction  windows  that  we  consider  have  rectangular  support, 
for  convenience: 


W-j  =  (m,n):{l£n£q,-p<m<p  &  n=0,l<m<p} 

causal 

(13) 

Wg  =  (m,n):{0<n<q,-p<m<p,  (m.nJ^O.O)} 

semi -causal 

(14) 

Q-  =  (m,n):{-q£n<q,-p<m<p,  (m,n)^(0,0)} 

non-causal 

(15) 

Wi  =  W.U(0,0)  ,  i=l  ,2 ,3 


(16) 
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The  prediction  region  S ^  in  (9)  is  also  called  the  non-symmetric  half 
plane  (NSHP)  in  the  literature  [6]. 

6)  The  predictor  coefficients  a(m,n)  in  (8)  must  be  chosen  according  to  some 
criterion.  Minimum  Variance  Predictors  are  designed  to  minimize  the 
variance  of  the  prediction  error,  i.e. 

62  =  min  E[e2(i,j)],  e(i,j)  =  u(i,j)  -  u(i,j)  (17a) 

For  Gaussian  random  fields,  the  predictor  will  be  linear.  The 
orthogonality  condition  associated  with  minimum  variance  criterion  is 

E{[u(i , j)  -  S  a(m,n)u(i-m,j-n)]u(i-k,j-£)}  =  B25(k,£)  , 

(m,n)€Sx 

(k,£)eSx  ,  x=l,2,3 

which  gives 

r(k,£)  -  £  ^  a(m,n)r(k-m,£-n)  =  B26(k,£)  ,  (k,£)eS.  ,  i=l,2,3  (17b) 

(m,n)eSi 

If  the  prediction  window  has  finite  support  Wi ,  1=1, 2, 3,  then  the  optimum 

A 

predictor  will  satisfy  (17b)  where  the  summation  is  over  (m,n)eWi,  i.e., 
r(k,£)  -  l  „  a(m,n)r(k-m,£-n)  =  B2S(k,£)  ,  (k,£)eS.  ,  1*1, 2, 3  (17c) 

m.neW.j 

Note  that  (17c)  must  be  satisfied  for  V(k,£)eS.o  W.  by  the  finite  support 
optimum  predictor. 

7)  A  Stochastic  Representation  for  a  Gaussian  field  (u(i,j)}  may  be 
written  as 

u(i,j)  =  u(1,j)  +  c(i,j)  (18) 

where  u(i,j)  Is  an  arbitrary  prediction  of  u(i,j),  and  (e(1,j)}  is 
another  random  field  such  that  (18)  realizes  the  covariance  proper¬ 
ties  of  (u(i,j)}.  Here  we  consider  minimum  variance  representations 
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(MVRs)  where  u(i,j)  is  chosen  to  be  a  causal,  semi-causal  or  non-causal 
minimum  variance  predictor.  The  difference  in  prediction  geometries 
determine  the  properties  of  the  prediction  error  field  (e(i,j)}.  As 
shown  in  [10],  (e( i , j ) }  is 

a)  a  white  noise  field  for  causal  MVRs. 

b)  white  in  the  causal  dimension  and  correlated  in  the  non-causal 
dimension  for  semi -causal  MVRs. 

c)  a  correlated  field  for  non-causal  MVRs. 

These  properties  are  important  because  they  point  out  that  for  semicausal 
and  noncausal  geometries  white  noise  driven  models  will  not  have  the 
minimum  variance  property. 

8)  A  MVR  as  in  (18)  can  be  considered  to  be  a  linear  difference  equation 
with  input  { e }  and  output  { u > .  The  SDF  of  {ul  is  then  given  by 


s£(z1  ,z2) 


r  A(z1  ,z2)A(z^  ,z2’) 


where  A(z1 ,z2)  = 


m  -n 


1  -  Z  A  a(m,n)z7  zZ 
(m,n)ESi 


(19) 

(20) 


is  called  the  prediction  error  filter  (PEF)  for  any  prediction  geometry. 
9)  The  MVR  of  (18)  is  stable  in  the  bounded  input,  bounded  output  (BIBO) 
sense  if  the  PEF,  A(z1 ,z2)  satisfies  certain  conditions.  For  the  three 
prediction  windows  in  (13-15),  the  MVR  is  stable  if  and  only  if 


a) 


A(z-|  ,z2)  t  0 
A(z1  ,z2)  f  0 
P 

A(z,  ,z« )  =  1  -  E 
c  m*l 


.  Iz^l, 

.  1  2-j  l=1  » 

a(m,0)zjm  - 


z2  = 


|z2l  -  1 


p  q 

E  E 

m=-p  n=l 


’  causal 
a(m,n)z^mz2n 


(21) 
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b)  A(z1  ,z2)  7s  0,  1  z-j  |  =1,  |z2l  >.  1  Semi-causal  (22) 

P  p  q 

A(z1,z2)  =  1  -  E  a(m,Q)z^m  -  E  E  a(m,n)z^mz2n 
m=-p  m=-p  n=l 

m^O 

c)  A(z-|  ,z2 )  ^  0  ,  { z-|  [  *  1,  |z2|  =  1  Non-causal  (23) 

p  q 

A(z,,z2)  =  1  -  E  E  a(m,n)zTmz2n 

m=-p  n=-q  c 

(m,n)^(0,0) 

10)  Closely  related  to  stability  is  the  concept  of  minimum  phase.  A 
stable  causal  filter  is  minimum  phase  if  it  has  a  causal  stable  in¬ 
verse.  A  stable  semi -causal  filter  is  semi -minimum  phase  if  it  has 
a  stable  semi -causal  inverse.  Marzetta  [7,8]  also  considers  analytic 
minimum  phase  (AMP)  filters  i.e.  minimum  phase  filters  that  are  also 

analytic  in  a  neighborhood  of  the  unit  circles. 

III.  ONE  DIMENSIONAL  NON-CAUSAL  MODELS 

First  we  present  some  results  on  1-0  non-causal  representations  with  regard 
to  properties  such  as  stability,  correlation  match,  etc.,  to  provide  direct  in¬ 
sight  into  similar  questions  for  2-D  representations.  The  1-D  non-causal 
representations  are  important  in  their  own  right  since  they  find  application  in 
data  compression  and  restoration  of  images  using  image  transforms  (for  example, 
see  [13,14]).  These  results  provide  certain  extensions  of  the  theory  of  non- 
causal  models  developed  in  [10]. 

A  non-causal  MVR  for  a  Gaussian  random  process  { u(k) >  may  be  written 

u(k)  =  u(k)  +  e(k)  (24) 


u(k)  =  E  a(£)u(k-0 

m 


(25) 
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where  u(k)  is  the  minimum  variance  prediction  estimate  of  u(k)  based  on  all  the 
past  and  future,  and  { e ( k ) }  is  the  prediction  error  process.  The  associated 
orthogonality  condition,  E[u(k)e(k)]=0,  implies  the  normal  equations 

r(n)  =  I  a(£)r(n-£)  +  B26(n),  Vn  (26) 

JtfO 

2  2 

where  r(n)  are  the  covariances  of  (u(k)}  and  8  =  min  E[e  (k)].  If  S(z)  is  the 
SDF  of  (u(k)}  then  the  above  equation  can  be  written  as 

S(z)  =  32/A(z)  ,  A(z)  ^  1  -  I  a(£)z"*-  (27) 

m 

where  A(z )  is  the  PEF.  The  solution  of  (27)  gives  the  parameters  of  the  non- 
causal  MVR  as  [10] 

aU)  =  a(-£)  =  -r"(t)/r'(0) 

7T 

82  =  1/r  (0)  =  1/D^-  S  ^(w)da)] 

-IT 

where  r"(£)  are  the  Fourier  series  coefficients  of  S”^(w).  Unlike  the  auto¬ 
regressive  (AR)  models  where  {e(k)}  turns  out  to  be  a  white  process,  in  the 
non-causal  MVR  it  is  non-white.  This  can  be  seen  by  multiplying  (24)  by 
e(k-n)  and  taking  expectations,  which  yields 

r  (n)  =  B26(n)  -  82  I  a(l)6(n-£)  or  S  (z)  =  62A(z)  (29) 

e  m 

Thus,  the  non-causal  PEF  is  not  a  whitening  filter  for  the  process  u(k). 

In  (25),  the  prediction  estimate  u(k)  uses  a[l_!  the  past  and  future  samples  of 
u(k)  in  the  prediction.  If  however,  (u(k)}  is  a  p*h  order  Markov  process  [15], 
then 

U( k)  |{u(k-£)  ,  =  u(k)  |{u(k-t)  ,  l<|£|£p}  .  (30) 

i.e.  all  the  information  needed  for  minimum  variance  non-causal  prediction  of 
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u(k)  is  contained  in  the  immediate  p  past  and  future  samples  of  u(k). 

Thus,  the  MVR  of  this  process  is  finite  order  with  {a(£)=0,  |£|>p)  . 
o 

The  other  parameters,  6  ,  (a(£),  |£|<p}  can  be  obtained  by  solving  a  finite 
subset  of  (26)  corresponding  to  |nj<p.  This  solution  will  also  satisfy  (26) 
for  |n)>p.  Eqn.  (26)  implies  { u >  and  {e^  are  uncorrelated  with  cross  covariance 
r  (k)  =  B  <$(k),  and  (29)  implies  that  (c>  is  a  moving  average  process  with 

Ut 

S  (z)  =  B2A(z);  A(z)  =1-1  a(£)z'£  (31) 

£=-p 

£*0 


Therefore,  the  SDF  realized  by  this  MVR  is  all  pole  and  rational.  Now,  if 
the  SDF  of  (u)  is  irrational  (but  well  behaved),  a  finite  order  MVR  will  not 
exist  for  the  process.  In  practice,  therefore,  one  approximates  the  infinite 
MVR  by  a  finite  order  minimum  variance  prediction  model .  The  considerations 

involved  in  designing  models  can  be  examined  in  relation  to  the  properties  of 

th  P 

finite  order  MVRs.  If  a  p  order  minimum  variance  predictor,  u(k)  =  l  a  (£)u(k-£) 

£=-p  P 

m 

is  used  to  predict  u(k),  and  ep(k)  is  the  prediction  error,  then  one  can  write 
u(k)  =  u(k)  +  £p(k).  The  associated  orthogonality  condition  gives 


r(n)  =  Z  a  (£)r(n-£)  +  S^6( n)  ,  |n)<p 

£=-p  v  y 

£7*0 


(32) 


2  2 
where  Bp  =  min  E[cp(k)] 

In  matrix  form,  the  above  equation  can  be  written  as 


Ra  =  gn, 
P~P  P“P 


(33a) 


where 


ap  -  C-ap(-p)  •••  -ap(-l)  1  -ap(l)  ...  -ap(p)]' 


(33b) 


V[o 


.0]' 


0 


1 


0 


(33c) 


-  13  - 


fr(0)  r(-l)  . r(-2pf 


k(2p) 


V(l)  r(0) 


The  solution  of  (33)  yields  the  coefficients 

6p  '  <34' 

v1  -  -ep[»;1]i.p*i  (35) 

If  Rp  is  positive  definite  (as  will  be  the  case  when  S(w)  >  0),  a  unique 
solution  exists.  Now  it  can  be  proven  that  the  SDF  of  the  residual  process 
(ep(k)}  is  given  by 

S£  (z)  -  [6p  +  Se  (z)]A  (z)  (36) 

P  P  v 

where  S  (z)  is  the  cross  SDF  of  {e  }  and  {u}.  In  general,  no  closed  form 
expression  can  be  found  for  S  (z),  and  the  residual  process  (c  )  is  not  simply 

V  p 

a  moving  average.  Thus,  the  order  non-causal  predictor  equation  realizes  the 
SDF  of  (u)  exactly  as 

Su(z)  =  l&Zp  +  S  (z)]/Ap(z)  .  (37) 

As  a  representation  for  (u),  the  exact  predictor  equation  is  not  useful  since 
the  forcing  function  (ep(k)}  is  inadequately  characterized.  A  more  practical 
model  for  {u}  is  obtained  by  constructing  a  process  (u)  with  finite  order  MVR 


u(k)  =  E  a  (fc)u(k-£)  +  c(k) 
£*-p  P 
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2 

where  {£}  is  chosen  to  be  a  moving  average  with  S~(z)  =  BpAp(z).  This  MVR 

O 

will  realize  the  rational,  all  pole  SOF  S~(z)  =  Bp/Ap(z),  and  can  also  be 
considered  as  a  model  for  {u}.  Leaving  aside  the  important  question  of  model 
accuracy  (for  the  time  being),  we  formally  write  the  definition  of  a  model, 
for  noncausal  MVRs. 


Definition:  A  p^  order  minimum  variance  non-causal  model  for  a  process 
{u( k) >  is  a  finite  order  MVR  for  another  process  (u(k)},  which  is  realized  by 

p 

solving  the  normal  equations  (33)  for  {ap(!l),Bp}  and  forcing  the  representation 
(38)  by  {e}  with  S~(z)  =  SpAp(z). 

With  models  being  formally  defined,  we  state  some  properties  of  these. 
Properties: 


1) 


2) 


3) 


4) 


ap(i)  =  ap(-i) 


i  -  1 ,2 , . . .  ,p 


>-l 


This  is  so  since  Rp  is  persymmetric  (symmetric  about  the  cross  diagonal) 
and  j>p  is  proportional  to  the  middle  column  of  R~*. 

2  2 
±  6P+1 

2 

i.e.  (Bp}  is  a  monotonically  non -increasing  sequence  with  increasing  order 
p.  This  is  intuitively  obvious  since  for  stationary  processes  inclusion 
of  more  samples  in  the  prediction  cannot  increase  the  prediction  error. 

If  {u}  is  P**1  order  Markov,  then  the  pth  order  non-causal  model  is  the  non¬ 
causal  MVR  of  the  process.  This  can  easily  be  seen  by  noting  that  the  SDF 
of  this  Markov  process  is  of  the  form  S(z)  *  C/Ap(z)  where  C  is  a  constant. 
The  inverse  transform  of  this  equation  is  of  the  form  of  (26).  The  subset  of 
this  equation  corresponding  to  |n|<p  is  exactly  the  realization  equation  of 
the  pth  order  model. 


.th 


The  definition  of  a  p  order  non-causal  model  implies  that  the  spectral 
estimate  from  the  model  can  be  taken  as 


SpU)  *  B*/APU) 


(39) 
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This  implies  that  the  non-causal  model  will  be  stable  if  and  only  if 

A  (z)  has  no  roots  on  the  unit  circle.  Unlike  the  causal  (AR)  model, 

P 

where  stability  of  the  model  is  equivalent  to  the  positive  definiteness 
of  the  covariance  sequence,  a  corresponding  result  has  not  yet  been 
discovered  for  non-causal  models. 

5)  The  correlation  matching  property  of  AR  models  does  not  hold  for  these 
non-causal  models.  In  fact,  examination  of  (33)  and  Property  1)  shows 
that  2p  covariances  are  used  to  calculate  p  model  coefficients.  Hence, 
given  a  set  of  admissible  model  coefficients,  there  exists  an  infinite 
number  of  covariance  sequences  that  will  satisfy  (33).  Also,  realization 
of  non-causal  models  does  not  require  spectral  factorization. 

The  questions  regarding  stability  and  correlation  matching  properties 
which  are  measures  of  the  goodness  of  the  model  are  answered  formally 
in  the  following  theorem. 

Theorem  1:  Let  (u(k)}  be  a  Gaussian  random  process  with  PA  SDF  S(z).  Then  the 
following  may  be  proven: 

a)  Convergence  of  PEFs:  The  sequence  of  non-causal  PEFs 

A  (z)  «  1  -  I  a  (l)z'Z 
H  Z=-p  H 

m 

converges  uniformly  in  some  neighborhood  of  |z|  =  1  to  an  analytic  limit 
PEF 


11m  A  (z)  =  A(z) 

P-KD  ” 


The  sequence  of  prediction  error  variances  converges  to  a  positive  lower 
limit,  3^ 
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b)  Stability:  i  pth  order  non-causal  model  is  stable  if  and  only  if 

Ap(z)  t  0  ,  | 2 |  =  1 

There  exists  some  pQ  <  °°  such  that 

Ap(z)  f  0  ,  |z|  =  1,  p  >  po  . 

c)  Correlation  match:  By  choosing  p  >  pQ,  the  difference  between  the  actual 
covariances  and  the  covariances  realized  by  the  model  can  be  made  arbi¬ 
trarily  small.  The  correlation  match  is  exact  either  at  p  -  “  or  at 

p  =  pQ  if  { u}  is  a  pQ  order  AR  sequence. 

The  proof  of  this  and  all  other  theorems  are  given  in  the  Appendix. 

Remarks:  Unlike  the  causal  (AR)  model,  it  is  not  true  in  general 

for  the  non-causal  models  that  stability  of  the  p^  order  model  guarantees 

s  t 

stability  of  the  (p+1)  order  model.  This  only  holds  true  for  P  >  PQ 

where  p  is  such  that  inf[A(w)]  -  e  >0  (see  A-14).  The  behavior  of  2-D 
0  po 

non-causal  models  is  similar  to  the  1-D  case,  and  the  method  of  analysis 
is  almost  parallel  and  is  presented  in  the  next  section. 

IV.  TWO  DIMENSIONAL  NON-CAUSAL  MODELS 

A  Gaussian  random  field  (u(i,j)}  has  the  non-causal  MVR 

u(i ,j)  *  u(i  ,j)  +  e(i  ,j)  (40) 

u(i,j)  =  E  a(m,n)u(i-m,j-n)  (41) 

(m,n)  eS3 

where  K(i,j)  is  the  minimum  variance  prediction  estimate  of  u(i,j)  based 
on  all  the  past  and  future,  and  e{(i,j)}  is  the  prediction  error  field.  From 
(17b)  and  in  analogy  with  (26) -(29)  for  the  one  dimensional  case,  we  have 

r(k,t)  *  Z  a(m,n)r(k-m,Jt-n)  +  626(k,fc),  V(k ,l)  (42) 

(m,n)  eS3 

S(z,,z2)  *  82/A(z1,z2)  ;  A(Zj ,z2)  =  1  -  E  a(m,n)zjmz2n  (43) 

(m,n)  eS3 
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Paralleling  the  discussion  leading  to  the  definition  of  1-D  non-causal 
models,  we  have  a  similar  definition  for  the  2-D  case. 

Definition:  A  (p,q)th  order  minimum  variance  non-causal  model  for  {u(k,£)> 
using  the  prediction  window  W3  is  a  finite  order  MVR  for  another  process 
{u(k,A)}. 

Realization :  Wi th 

u(i,j)  =  z  ^  a  (m,n)u(i -m,j-n)  +  e  Q(i,j)  (47) 

(m,n)e^3  p’q  P,q 

2  2 

the  {a  (m,n)}  are  chosen  to  minimize  E[e  k , e)3  =  6  .  The  associated  ortho- 

P»H  P»H 

gonality  condition  yields  the  realization  equation 

r(k,£)  =  E  a  (m,n)r(k-m,2-n)  +  8*  5(k ,£) ,  (k,£)eW-  (48) 

(m,n)cW3  p*q  p,q  3 

In  matrix  notation,  equation  (48)  can  be  written  as 

2 

Rp,q^p,q  =  Bp,<A>,q  (49a) 

where  R  is  a  doubly  block  Toeplitz,  symmetric  matrix  defined  as 
P 


-  18  - 


R  - 
Rk 


r(-2p,k)" 


(49b) 


^p.q 


-  C-L 


4 


4] 


^  [-a(-p ,k)  -a(-p+l,k)...  -a(0,k)  ...  -a(p,k)]T  (49c) 

a(0,0)  £  1  (49d) 

ll  „  =  tOT  °T  ll  0T  ...  0T]  ,  1  =  [0  0  ...  0  1  0  ...  0]T  (49e) 

p  ,q  —  — p  p 

(q+l)st  entry  (p+l)St  entry 

If  S(w.,oj9)  >  0  then  R  is  positive  definite,  V(p,q)  and  a  unique  solution 

i  c  p  9q 

exists  for  the  prediction  coefficients  in  (49a)  given  by 


=  1/[R 


-1 

p,q]k,k  ’ 


W”'"’  '  •6p.q[Rp!<l]t.k 


k  =  q(2p+l)+p+l 


;  l  =  (n+q) (2p+l)+m+p+l 


(50a) 

(50b) 


Again,  once  the  model  coefficients  are  calculated,  the  field  {£( i , j ) >  with 
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then  the  (p,q)  order  non-causal  model  realizes  this  SDF  exactly. 

In  other  words,  it  is  the  non-causal  MVR  of  the  field,  satisfying 
the  orthogonality  condition  over  S.j.  In  general,  the  (p,q)  order 
non-causal  model  can  be  expected  to  realize  an  arbitrary  SDF  only 
approximately.  This  is  due  to  the  fact  that  orthogonality  is  satis¬ 
fied  only  over  the  window  W3  which  is  a  subset  of  S3-  Increasing 
the  order  of  the  models  may  therefore  be  expected  to  give  better 
correlation  match  and  spectral  fit. 

The  following  theorem  shows  finite  order,  stable  MVRs  can  be  realized  to 
achieve  arbitrarily  close  correlation  or  spectral  match. 
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Theorem  2:  Let  {u(k,£)}  be  a  Gaussian  random  field  with  PA  SDF  S(zj,z,). 
Then  the  following  may  be  proven: 

a)  Convergence  of  PEFs:  The  sequence  of  non-causal  PEFs 


'Wzl-z2>  =  1 


x  „ 

m.neW, 


Vc,<"'n)zl 


-m_  -n 


converges  uniformly  on  |z^|  =1,  | j  =  1  to  the  analytic  limit  PEF 
A(z1,z2),  i.e. 


lim  A  (z^.Zp)  =  A(z.,zJ  ,  |  z,  |  =  |zJ 

(P.q)—  P,q  1  2  12  1  2 


The  sequence  of  prediction  error  variances  (£T  }  converges  monotonical ly 

P 

2 

to  a  positive  lower  limit,  6  . 


lim  6 

(p.q)-* 


p.q 


02  =  ll  Cl  .w^du^dw,]-1 


4tt‘ 


J1  *2  UUJ1  2  “ 


b)  Stability:  A  (p,q)  order  non-causal  model  is  stable  if  and  only  if 


Vq<ZrZ2>  ‘  0  •  lZll 


I  z2 !  ■  1 


There  exists  some  (p0,qQ)  <  00  such  that 


Ap,q(zl’z2)  f  0  ’  izl!  =  lz2>  =  l,  (P*q)>(P0*qo) 


c)  Correlation  match:  By  choosing  (p.q)  >  (p  ,q  ) ,  the  difference  between 
the  actual  correlations  and  the  correlations  realized  by  the  model  can 
be  made  arbitrarily  small.  At  (p.q)  =  =°,  the  correlation  match  is  exact. 


Remarks-  The  above  theorem  is  similar  to  the  theorem  for  1-D  non-causal 
models  except  for  one  difference.  For  the  2-D  case,  we  have  been  able 
to  prove  uniform  convergence  of  the  PEFs  to  the  limit  PEF  only  on  the  unit 
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circles,  whereas  for  the  1-D  case,  uniform  convergence  is  obtained  in  a 
neighborhood  of  the  unit  circle.  This  is  due  to  the  complexity  introduced 
by  the  added  dimension.  In  any  case,  the  result  is  still  general  since 
for  stability  and  correlation  matching  considerations,  only  convergence 
on  the  unit  circle  is  required. 

V.  SEMI -CAUSAL  MODELS 

Semi -causal  models  are  recursive  in  one  of  the  dimensions  and  non¬ 
recursive  in  the  other.  In  the  following,  we  investigate  methods  of 
realizing  finite  order  semi -causal  models  that  will  match  a  given  SDF 
arbitrarily  closely.  The  realization  equations  may  be  found  in  [10], 
but  we  present  them  here  for  completeness.  Unlike  non-causal  models 
which  do  not  require  spectral  factorization  for  realization,  semi-causal 
models  are  realized  by  factorization  of  S(zj,z2)  in  one  of  the  variables, 
corresponding  to  the  causal  dimension.  The  two  methods  of  factorization 
that  we  consider  are  distinct  from  one  another.  In  the  first  method,  the 
SDF  on  its  entire  region  of  support  is  used  to  realize  the  model.  We 
present  some  theorems  and  use  a  Levinson  type  computational  algorithm  [8] 
to  obtain  rational  models. 

In  the  second  method,  covariances  given  on  finite  windows  are  used  to 
design  finite  order  models  by  solving  a  set  of  linear  block  Toeplitz 
equations.  This  is  similar  to  the  2-D  non-causal  case  and  similar  con¬ 
vergence  results  for  the  models  are  proven. 

Characterization  of  Semi -causal  MVRs: 

The  semi-causal  MVR  for  a  Gaussian  random  field  (u(i,j)}  can  be  written  as 

CO  00  oo 

u(i,j)  =  l  a(m,0)u(i-m,j)  +  l  l  a(m,n)u(i-m,j-n)  +  e(i,j)  (51) 
m=-«>  n=l  m*-00 

m^O 


The  orthogonality  condition  (17b)  becomes 
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OO  GO 


r(k,£)  =  l  a(m,0)r(k-m, C.)  +  l  l  a(m,n)r(k-m, ?-n)  +  8  6(k,0.  k^O.Ve  (52) 
m=-“  n=l  m*-<» 

m^O 


Defining 


a0(z,)  =  1  ‘  E  a(m»°)zi 
L  m=-°° 

m^O 


-m 


(53a) 


It  can  be  shown  that  a(m,0)  form  a  positive  definite  sequence,  and  that 

_  1 

a(m,0)  =  a(-m,0).  This  implies  that  ag"A(z^)  exists.  Hence  we  may  define 


an(z1)  =  l  a(m,n)z“n  ,  a^Zj)  =  an(z1)/aQ(z1)  ,  n>l 
"  *  m=-°° 


(53b) 


and  taking  z-transform  on  both  sides  of  (52)  w.r.t.  k,  we  obtain 


re(zl}  =  I  an(zI)rt-n(zl)  +  62ag1(z1)5(t)  ,  t>0  (54) 

n=l 


In  matrix  form,  the  above  can  be  written  as 


1 

btzjf 

-al(zl> 

0 

0 

« 

b(zx)  -  B^q^Zj) 
(55) 


In  the  following  we  prove  that  an  SDF  may  be  uniquely  factored  as 


S(z j jZg )  - 


s2a0(zl> 

A(Zj  .ZgMU^  ,Zg  ) 


(56a) 
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where 

A(z1,z2)  =  1  -  Y.  a(m,n)zjmz2n  .  a^Zj)  =  A(zi>°°)  (56b) 

(m,n)eS2 

Then  we  show  a  computational  procedure  for  solving  (55)  recursively  to 
obtain  the  factorization. 


Theorem  3:  Let  the  SDF  S(z1>z2)  be  PA.  Then  it  may  be  uniquely  factored  as 
in  (56),  where  the  PEF  A^.z,,)  is  semi-minimum  phase. 

The  following  theorem  gives  a  computational  procedure  similar  to  the  1-D 
Levinson  algorithm  which  recursively  solves  (54)  to  obtain  the  semi-causal 
factors  guaranteed  in  Theorem  3. 


-  24  - 


Theorems  3  and  4  provide  the  theoretical  basis  for  designing  semi -causal 

MVRs  given  the  SDF  of  the  random  field.  The  theorems  also  point  the  way  to 

obtaining  stable,  finite  order  models  from  the  generally  irrational  MVRs  guaranteed 

by  the  theorems.  Rational ization  is  achieved  in  two  steps:  First,  given  the 

(r£(Zi)},  the  recursions  in  (57)  are  run  to  a  value  of  q  such  that  the  PEFs 

obtained  approximate  the  SDF  to  a  desired  degree  of  accuracy.  The  behaviour 
2 

of  the  sequence  { }  can  be  used  as  an  indicator  of  the  goodness  of  spectral 

2 

fit.  The  recursions  can  be  stopped  when  the  decrease  in  becomes  marginal, 
thereby  determining  the  finite  order  in  the  Zg  variable.  However,  the  PEFs  are 
still  irrational  in  the  z^  variable. 
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As  far  as  the  second  stage  of  approximation  is  concerned,  an  attempt 
to  rationalize  the  coefficients  ^an(z^)^  will  in  general  lead  to  unstable 
models.  This  can  be  circumvented  by  examining  the  recursions  of  (57),  which 
shows  that  as  long  as  the  sequence  of  analytic  functions  (Pn(z^)}q  have 
magnitude  less  than  unity  on  |z^|  =  1,  the  filters  ^(Zj.Zg)  are  guaranteed  to 
be  AMP.  It  is  possible  to  find  polynomials  fp^Zj)}  with  magnitude  less  than 
unity  on  |z^|  =  1  which  uniformly  approximate  the  analytic  functions  { pn ( z ^ ) } . 
For  example,  a  truncation  and  windowing  (by  Wn ( k ) )  to  a  suitable  length  of  the 
Fourier  coefficients,  (pn(k)}  of  { pn (oj^ ) }  exhibit  such  behaviour.  We  adopt 
this  procedure  for  rationalization.  Also,  in  [8],  it  is  shown  that  if  S{Zj,z.,) 
is  PA,  then  the  reflection  coefficient  sequence  is  exponentially  bounded,  i.e. 

|  pn  ( k )  |  <  Aa^o^l  ,  n>0,  Vk  ,  Ckctj.a^l 

This  property  allows  us  to  truncate  Pn(k)  to  shorter  and  shorter  lengths  as 
n  increases,  while  retaining  stability.  The  following  portions  of  (57)  allow 
us  to  reconstruct  the  finite  order  model: 


an,0^zP  =  1  ’  n=0’1’2,,"’q 

(a) 

Sn,i*zl*  =  an-l,i(zl^  '  °n^zl)an-l,n-i^zl  ** 

i  =  l , . . .  ,n-l 

(b) 

an,n<zl>  * 

(c) 

b0(zi)  -  r0 ( z i ) 

(d) 

=  6n-l(zl),:i-pn(2l)pn(zi1):i 

(e) 

pn  _k 

where  p^(Zj)  =  E  pn^k^zl 

(f) 

k=-Pn 


(59) 


Ultimately,  however,  we  are  interested  in  the  actual  model  coefficients, 
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ar  .  =  ar  n(z1)ar  .  (z.).  It  now  remains  to  find  a  rational  approximation  to 
q » 1  q ,u  1  q *  i  I 

b  (z,)  which  will  yield  a|"  ^(z,).  Since  b  (z,)  is  a  PA  function,  we  can  fit 
q  1  q  ,U  1  q  l 

a  finite  order  1-D  non-causal  model  (see  Section  III)  that  will  approximate 
bqU^)  to  the  desired  degree  of  accuracy,  i.e. 

Vzi>  -65(zi>  *  sX,o(2i> 

This  gives  the  desired  finite  order  model  that  is  semi-minimum  phase.  Finally, 
it  must  be  mentioned  that  the  region  of  support  of  the  predictor  coefficients 
will  almost  never  be  rectangular  due  to  the  convolutions  involved  in  the  re¬ 
cursions  [see  Eqns .  (57)].  Of  course,  this  does  not  pose  any  limitation. 

As  an  example,  the  predictor  geometry  is  shown  in  Fig.  2  for  q=3,  with 
the  sequences  (pj(k) }  ,{p2(k)} ,  and  { p3( k) }  truncated  to  p^,  p2=2,  P3=l 
[see  (59f )] ,  respectively,  and  for  a  non-causal  model  order  of  p=3. 

A  flow  chart  of  the  rationalization  procedure  is  shown  in  Fig.  3  for 
convenience. 

The  above  method  of  obtaining  stable,  semi -causal  MVRs  assumed  complete 
knowledge  of  the  2-D  covariance  sequence  in  one  of  the  dimensions.  The  question 
we  address  next  is  the  problem  of  realizing  stable  semi -causal  MVRs  given 
correlations  on  a  finite  2-D  window.  Except  for  the  difference  in  prediction 
geometry,  the  methods  involved  are  similar  to  the  2-D  non-causal  case. 

For  a  prediction  window  such  as  W2>  the  semi -causal  minimum  variance  model 
can  be  written  as 

u(i,j)  =  Tj(  i » j )  +  e  _(i,j)  (6°) 

K 

n(i  ,j)  -  Z  „  an  n(m,n)u(i-m,j-n)  (61) 

(m,n)eW2  p,q 

2  2 

with  min[E(e„  „  (i,j))]  =  8_  _  .  The  normal  equations  (17c)  written  for  the  region 
P»H  P  »H 

W9  give 

P  P  Q  2 

r(k,£)  =  Z  an  (m,0)r(k-m,£)  +  Z  Z  a  (m,n)r(k-m,t-n)+6  a6(k,t);(k,l)eH- 
m=-p  p,q  m-p  n=l  p’q  p,q  2 

m^O 


(62) 


e)  Prediction  Geometry  for  the  Semlceueel  Model 


Fig.  2:  Continued 
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First  Stage  of  Rationalization 


Second  Stage  of  Rationalization 


Fiq.  3:  A  Method  of  Obtaining  Finite  Order  Semicausal 
Models  Via  Levinsons  Recursions 
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Defining  a.  10,0)  4  1.  we  .ay  write  the  above  egoation  in  -trix  for.  as 
P 

-  R2  1 

Rp,q%>.q  ep,q-p,q 


(62a) 


where 


(62b) 


R 


k 


4 


(62c) 


T 

%>,q 


T  J  .  aT]  .  ,  -  [-a(-p.k)  -a(-P+l,k)  ...  -a(0,k)  ...  -a(p,k)]T  (62d) 


•Cfioii 


jT  =  [1T  0T  0T..0T]  ;  1  =  [0  0  ...  o  1  0  ...  o] 

-Vq  L-p  -  -  ~  ^  /-.iSt 


(p+1  entry) 


(62e) 


and  6Z  >  0  exists  for  (62a)  given  by 

a  p,q 

8p,q  =  1/tRp.qV1»P+1 


(63a) 


;  k  =  n(2p+l)+m+p+l 


(63b) 


ap,q(m,n)  ep,q^Rp,q^k,p+l 

From  equations  (62),  it  is  evident  that  covariances  from  a  window  twice 

the  size  of  W9  are  needed  to  calculate  (a  (m,n)}  on  W9.  Hence,  there  is  no 
c  p  ,q  d 

unique  correspondence  between  the  predictor  coefficients  and  the  covariances 
realized  by  the  model.  Also,  stability  is  not  guaranteed.  These  statements 
were  true  for  non-causal  models  realized  by  this  method,  and  as  we  shall  see, 
will  be  true  for  causal  models  also.  Theorems  similar  to  Theorem  2  may  be 
proven  for  these  mclels  also.  The  method  of  proof  is  similar,  with  minor  dif¬ 
ferences  arising  from  different  prediction  geometries. 


Theorem  5:  Let  the  Gaussian  random  field  (u(k,«.)}  have  a  PA  SDF,  Sfzj.z-). 
Then,  the  following  may  be  proven: 

a)  Convergence  of  PEFs:  The  sequence  of  semi -causal  PEFs 


Ap,q(zl’Z2)  =  1  - 


£  , 

(m,n)eW9 


converges  uniformly  in  the  region  =  {|Zj|  =  1,  | z^ I  i  1)  to  the  ana 
lytic  semi-minimum  phase  PEF  Afz^^). 

2 

The  sequence  of  prediction  error  variances  (6  }  converges  mono- 

P  »H 

2 

tonically  to  a  positive  lower  limit,  B 

lim  Bp  =  B2  =  1/C^r  d'^co^dwj]  ,  d(w1)  =  82a‘1(w1) 

(p,q)-«o 


b)  Stability:  A  (p,q)  order  semi-causal  model  is  stable  iff 

Ap,q(zl,z2^  ^  °’  ^zl,z2^  e  Z1 


There  exists  some  (Pg^)*  00  such  that 
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c)  Correlation  match:  By  choosing  (p,q)  >  (po,qQ),  the  difference  between 
the  actual  correlations  and  the  correlations  realized  by  the  model  can 
be  made  aribtrarily  small.  At  (p,q)  =  °°,  the  correlation  match  is  exact. 

VI.  2-D  CAUSAL  MODELS 

Causal  models  are  recursive  in  both  dimensions  with  an  ordered  definition 
of  past  and  future.  Realization  of  these  models  from  the  SDF  S(Zj,Zo)  requires 
factorization  in  both,  z^  and  z ^  variables.  Similar  to  the  semi-causal  case, 
we  consider  two  methods  of  obtaining  these  models.  Realization  of  these  models 
from  the  SDF  on  its  entire  region  of  support  using  the  LP  approach  has  received 
attention  in  the  literature  [7-8].  Obtaining  rational  models  has  been  considered 
in  [8]  by  way  ce  finite  support  reflection  coefficient  representation.  The 
method  presented  here  rationalizes  the  reflection  coefficients  obtained  in 
the  2-D  Levinson  recursions  to  obtain  finite  order  models.  The  other  method 
of  obtaining  rational  models  is  to  use  covariances  given  on  a  finite  window, 
and  solve  a  finite  set  of  linear  equations.  Convergence  and  stability  results 
for  models  realized  by  this  method  are  similar  to  the  non-causal  and  semi-causal 
results . 

Characterization  of  Causal  MVRs: 

The  causal  MVR  for  a  Gaussian  random  field  {u(i,j)}  is 

OO  00  00 

u(i , j)  =  l  a(m,0)u(i-m,j)  +  l  l  a(m,n)u(i-m,j-n)  +  e ( i , j )  (64) 

m=l  n=  1  m=-°° 

The  corresponding  normal  equations 

00  OO  OP  2 

r ( k , 5. )  =  l  a(m,0)r(k-m,s>)  +  l  l  a(m,n)r(k-m,e-n)  +  6^6{k ,2)  ,  (k,fc)eS1  (65) 
m=l  n=l  m3-® 

reduce  in  the  z^-transform  domain  to 
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r^(zl)  =  n^1a'n(zl)rS.-n(zl)  +  32a0(21)ao1{zj1)6(e)  ,  £?0  (66) 

where  an(Zj),  a  (Zj);  n=l,2,...  are  defined  in  (53b)  and 

a0Ui )  =  1-1  a(m,0)z"m  (67) 

m=l 

The  proof  of  the  existence  of  a  unique  solution  to  (66),  which  is 
analytic  minimum  phase  is  similar  to  Theorem  3. 

The  procedure  given  in  Theorem  4  can  be  directly  used  to  find  the  causal 
MVR  except  for  the  following  differences: 

1)  (an  g(zj)}  are  minimum  phase  functions  obtained  by  a  second  stage  of 
factorization  in  the  z^  variable  of  the  positive  analytic  functions 
(bn(z1)}  such  that 

br,(zI>  '  8X.0(zl>,n.0(2i1)  <68) 

2)  The  PEFs  {An(zj,z2)}  defined  as 


Vzl>22>  '  Vo!zl)Sn(zl'z2> 


are  now  analytic  minimum  phase. 

3)  The  rationalization  procedure  is  the  same  as  for  semi -causal  models, 
except  that  a  1-0  causal  model  is  used  to  fit  bq(z^)  as  opposed  to  a 
non-causal  model . 


Now  we  consider  the  problem  of  finding  finite  order  causal  models,  given 
covariances  on  a  finite  window.  The  normal  equations  for  a  (p,q)  order  causal 
model  is 

r(k,0  =  f  an  (m,0)r(k-m,£)  +  f  \  a  (m,n)r(k-m,£-n)  =  6Z6(k,£),  (k,fc)eW.  (70) 
m=l  p’q  m»-p  n=l  p,q  1 


-  31 


In  matrix  notation,  the  above  equation  can  be  written  as 

Rp,q^p,q  "  3p,pip,q  (71a) 

where 


Rk  is  defined  in  (62c)  and 

r(p,k)  r(p-l,k) 

Rk  -  ; 

r(2p,k)  r(2p-l,k) 


r(-p,k) 


r(0,k) 


,  R;k  =  R'T  (71b) 


T  4  r  T  T  T 

Sp.q  [5o  -2 


4 


a0  '  t1  -aP,q<‘-0> 


-apjq(p,°): 


dp,q(0’°>  4  1 


4  4  C-ap,qt-P.f) 


•ap,q(°.’) 


-ap,q(P.<>] 


i!  =  qI  ot  ot 

-p  ,q  L~ p  -  - 


0  ]  ,  1^  =  [1  0  .  0] 


(71c) 


Given  that  Sfwj.w^)  >  0,  a  unique  solution  to  the  system  of  equations 
(71)  exists.  Considerations  such  as  convergence  of  PEFs,  stability  and 
correlation  match  follow  similar  lines  of  reasoning  as  in  the  non-causal  and 
semi -causal  case.  We  briefly  state  the  results: 
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1)  The  sequence  of  PEFs 


Ap,q*zl,z2* 


1  -  l  ap,q(m‘0)zim  '  ^  I  ap,q(m’n)zimz2n 

m=l  p,M  m=-p  n=l  P,H 


converge  uniformly  to  the  limit  PEF  A(Zj,z2)  in  the  region 


Z  =  { | Zj |  =  1,  \z2\  >  1}  U  { | Zj |  >  1,  z2  =  »} 


The  sequence  of  prediction  error  variances  converges  monotonically 
to  a  positive  lower  limit 

2  2  1  71 

lim  3_  n  =  B  =  exp[ — ~  ff  log  S(to,  ,w,)dw1  ,da),] 

(p,q)~°  P*q  4ttZ  -tt  12  12 

2)  For  some  (p,q)  >  (pQ,qQ),  Ap,q(zi«z2^  ?  0  for  (zi*z2^  e  z«  so  that 
the  causal  models  of  order  (p,q)  >  (pQ,qo)  are  stable. 

3)  For  (p,q)  >  (pQ,qo)  the  correlation  match  can  be  made  arbitrarily  small. 

VII.  EXPERIMENTAL  RESULTS  AND  COMPARISONS 

The  following  provides  a  verification  of  the  various  theoretical  results 
obtained  for  2-D  stochastic  models  in  the  preceding  sections.  The  results  are 
demonstrated  for  a  covariance  model  that  is  often  used  in  image  processing  viz; 
the  exponential  isotropic  covariance,  r(k,Jl)=p  kZ+*-2  .  For  our  simulations,  we 
have  used  p=0.9.  A  5x5  portion  of  this  covariance  sequence  is  shown  in  Fig.  4. 
Only  the  entries  in  the  positive  quadrant  are  given,  since  the  other  quadrants 
can  be  filled  in  by  symmetry.  The  SDF  corresponding  to  this  covariance  sequence 
is  irrational  and  satisfies  the  PA  assumption.  Wherever  required,  z-transforms 
were  evaluated  on  the  unit  circle(s)  using  256  point  OFTs  in  1-D  and  256x256  DFTs 
in  2-D. 

Non-causal  models  were  realized  for  this  covariance  sequence  by  solving 
the  linear  block  Toeplitz  equations  of  (49)  using  an  efficient  recursive 
algorithm  [16].  The  SDF  corresponding  to  a  (p,q)  order  model  is 
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Fig. 
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Covariances  Corresponding  to  r(k,H) 
on  the  Quarter  Plane 
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Sp,q(zl,Z2) 


Bo.q 


P  q 

where  An  q(z1’Z2)  =  1  '  l  l  ap,q(m*n)zimz2n  • 

m=-p  n=-q  K,M 
(m,n)?«(0,0) 

The  covariances  realized  by  the  model,  {r  ( k , £ ) }  are  obtained  as  the 

P 

inverse  transform  of  Sp  ^(Zj^)  • 

Figure  5  shows  the  PEFs  and  the  covariance  mismatch  for  (1,1)  and  (1,2) 

order  non-causal  models.  Examination  of  the  PEF  for  the  (1,1)  model  seems  to 

suggest  a  four  point  model,  but  this  is  belied  by  the  substantial  covariance 

2  2 

mismatch  obtained.  The  (1,2)  model  gives  a  smaller  mismatch,  and  B1  2  <  ^ 

Results  obtained  for  progressively  larger  models  confirm  the  results  of  Theorem  2. 

Infinite  order  spectral  factorization  was  carried  out  to  obtain  the  semi- 
causal  MVR  using  the  Wiener-Doob  factorization  method.  Details  of  this  method 
can  be  found  in  [10].  The  coefficients  of  the  PEF  corresponding  to  the  MVR  are 
shown  in  Fig.  6.  Examination  of  the  coefficients  U(m,0)}  indicate  that 
e{(i,j)}  is  very  nearly  a  first  order  moving  average  process  in  the  non-causal 
dimension. 

Semi-causal  models  were  realized  using  the  rationalization  procedure  of 
Section  V.  Fig.  7  shows  the  PEF  of  the  rational  model  obtained  by  using 
second  order  in  the  causal  dimension  (q=2).  The  infinite  length  reflection 
coefficient  sequences  {pj(k)}  and  { p2 ( k ) }  were  truncated  to  5  terms  and  3  terms, 
respectively,  i .e. 


Pj(k)  ,  | k| <2 

0  ,  o.w. 


P2(k)  ,  |k|<l 

0  ,  o.w. 


A  second  order  (p=2)  1-D  non-causal  model  was  used  to  fit  ^(zj),  giving 
a  total  of  23  coefficients  for  the  semi-causal  model.  For  this  case,  the  PEF 
is  given  as 
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Fig.  7:  Rationalized  Semicausal  Model  Corresponding  to  q=2,  { p1 ( k ) } 
Truncated  to  5  Terms,  { P2<k) }  Truncated  to  3  Terms  and  p=2. 
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A(21,z2)  =  1  -  Z  a(m,0)zjm  -  Z  afm.Dzj^1  -  E  a(m,2)zjmz22 
m=-2  m=-5  m=-3 

m^O 

The  SDF  realized  by  this  model  is  then 

2 

S(z,,z2)  =  B2(l  -  Z  a(m,0)zJm)/A(z1>z2)A(zJ1,z21) 
m=-2 
m^O 

The  coefficients  {a(m,0)}  can  be  seen  to  differ  appreciably  from  the  'true' 
coefficients  in  Fig.  6,  which  explains  the  substantial  covariance  mismatch 
in  Fig.  7. 

By  using  q=l,  truncating  (pj(k)}  to  15  terms  and  using  p=2  yielded  the 

semi -causal  PEF  in  Fig.  8,  having  a  total  of  24  coefficients.  As  can  be 

seen,  the  covariance  match  here  is  much  better  than  in  the  previous  case. 

2 

Also,  the  coefficients  {a(m,0)}_2  are  closer  to  the  true  coefficients.  This 
emphasizes  the  importance  of  the  leading  reflection  coefficients  which  have 
the  larger  magnitudes.  Since  [see  (59)]. 


Vzl)  *  V^iSl^1  "  bi(zi)p^zi1)3  -  6q/*q#o^zl^ 


the  coefficients  (a(m,0)}  will  be  most  sensitive  to  (p^(k)}.  However,  the 
price  paid  is  large  order  models. 

Semi-causal  models  are  also  obtained  by  solving  the  set  of  linear  block 
Toeplitz  equations  (62).  Fig.  9  shows  the  semi-causal  PEFs  obtained  for 
model  orders  (p,q)  =  (1,1),  (2,2),  (3,3).  Examination  of  the  PEFs 
for  orders  (1,1),  (2,2)  and  (3,3)  shows  that  the  PEF  coefficients  converge 

quite  rapidly  to  the  true  coefficients  in  Fig.  6.  Also,  3  K  B2  Z  <  B\  l 

2  1  *  * 
and  approach  the  limit,  6  of  the  infinite  PEF.  The  correlation  match  also 

gets  better  and  these  are  shown  in  Fig.  10. 
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Fig.  8:  Semicausal  Model  Obtained  Using  q=l,  Truncating 
{pj(k)}  to  15  Terms,  and  p=2. 
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Fig.  10:  Covariance  Mismatches  of  the  Semicausal 
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Next,  for  causal  models,  the  PEF  corresponding  to  the  infinite  order 
MVR  is  shown  in  Fig.  11.  This  has  been  obtained  by  Wiener-Doob  factorization, 
[10] .  Causal  models  were  realized  using  the  rationalization  procedure  similar 
to  Section  V.  Recall  that  the  procedure  is  the  same  except  that  instead  of 
fitting  a  non-causal  model  to  ^(z^,  as  in  the  semi -causal  case,  a  further 
factorization  of  bq(z^)  into  minimum  phase  and  maximum  phase  factors  is  involved 
in  the  causal  case.  Fig.  12  shows  the  PEF  of  the  model  obtained  by  using  q=2. 
truncating  (p^k)}  and  (p2(k)}  to  5  and  3  terms,  respectively,  and  fitting  a 
second  order  (p=2)  1-D  causal  model  to  ^2 ( 2 1 ) . 

The  PEF  has  17  coefficients  and  realizes  the  SDF  given  by 

SUpZg)  =  82/A(z1,z2)A(zj1,Z21) 

2  5  3 

A(Zl,z2)  =  1  -  Z  a(m,0)zjm  -  I  a(m,l)z”mz21  -  I  a(m,2)zjmz22  . 

m=l  m~-3  m=-l  *  *" 

From  Fig.  12(f)  the  covariance  mismatch  is  seen  to  be  quite  high,  though 

it  is  less  than  the  corresponding  semi-causal  case  (Fig.  7(b)).  Fig.  13 

shows  the  causal  PEF  coefficients  corresponding  to  q  =  1,  (p^k)}  truncated 

to  15  terms  and  p  =  5.  The  total  number  of  coefficients  is  26.  The  correlation 

mismatch  is  much  less  than  the  previous  case,  and  better  than  the  corresponding 

semi-causal  case  (Fig.  8(b)).  Considerations  of  the  effects  of  reflection 

coefficient  truncation  are  similar  to  the  semi-causal  case. 

The  PEFs  obtained  by  solution  of  the  linear  equations  of  (88)  corresponding 

to  causal  model  orders  of  (p,q)  s  (1,1),  (2,2),  and  (3,3)  are  shown  in 

Fig.  14.  This  again  verifies  the  convergence  of  the  PEFs  (Ap  ^(z^.Zg)}  ancl 

2 

prediction  errors,  (8„  „}  to  their  limiting  values  in  Fig.  11.  Fig.  15 

P  *H 
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Fig.  11:  Infinite  Order  Causal  PEF  Obtained  by 
Wiener-Doob  Factorization 
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Fig.  13:  Rationalized  Causal  Model  Obtained  Using  q=l, 
(Pj(k)}  Truncated  to  15  Terms  and  p  =  5 
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Causal  Model  PEFs  Obtained  by  Solving 
Finite  Order  Equations 
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shows  the  covariance  mismatches  obtained  with  the  above  three  causal  models. 

The  (1,1)  causal  model  seems  to  provide  a  surprisingly  good  covariance  match 
compared  to  the  corresponding  semi-causal  model  (Fig.  10(a)). 

VIII.  CONCLUSIONS 

We  have  proved  a  new  theorem  on  the  properties  of  1-D  non-causal  models 
which  establishes  convergence  of  PEFs,  and  guarantees  model  stability  and 
improvement  in  covariance  matching  properties  after  a  finite  model  order. 

The  spectral  factorization  methods  of  [7,8]  (for  causal  models)  and  [10] 
(for  causal  and  semi-causal  models)  realize  irrational  models  by  the  solution 
of  an  infinite  set  of  normal  equations.  Theorems  were  proven  to  establish 
existence  and  uniqueness  of  solutions.  The  SDFs  realized  by  the  Levinson 
like  computational  algorithm  of  [7,8]  were  shown  to  converge  uniformly  to  the 
true  SDF.  A  method  of  rationalizing  the  reflection  coefficients  obtained  ir, 
this  algorithm  was  used  to  obtain  rational  models  with  guaranteed  stability. 

A  method  of  realizing  2-D  non-causal,  semi-causal  and  causal  models  by 
solving  a  set  of  linear  equations  [10]  was  considered.  The  advantage  of  this 
method  is  that  only  a  finite  set  of  equations  need  be  solved.  However,  these 
models  were  not  guaranteed  to  be  stable  or  provide  convariance  match.  The 
results  proven  in  this  paper  have  shown  the  uniform  convergence  of  the  PEF 
sequence  to  a  limit  PEF,  from  which  model  stability  and  improved  covariance 
match  were  shown  to  follow  after  some  finite  model  order.  These  results  enhance 
the  practical  appeal  of  this  technique. 

Examples  of  finite  order  model  realizations  were  shown  for  an  isotropic 
covariance  function,  which  corroborated  the  theory  developed.  For  this 
covariance  function,  it  was  found  that  the  non-causal  model  needed  much  larger 
model  orders  than  either  the  causal  or  semi -causal  models,  to  provide  uniformity 


-  37  - 


In  covariance  matching  results.  The  performance  of  the  latter  two  models  were 
quite  good  and  on  par  with  each  other. 
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APPENDIX 


Definitions:  The  following  definitions  and  properties  are  useful  in  proving 
the  theorems  presented  in  this  paper. 

a)  For  a  vector 

x  =  [x(l)  x(2)  x(3)  _  x(N)]T 

the  Frobenius  norm  is  defined  as 


If  A  is  a  M  x  N  matrix,  then 


|| A  ||F  ^  Jl  Z  a2(i,j) 
h  i=l  j=l 

It  can  be  proven  that 

l|Ax||F  <  ||A||f||x||f 

b)  The  l_2  or  2-norm  of  x  is 

Hill  4/s  x2(i)  -  |ii||F 
£  i=l 

If  A  is  a  M  x  N  matrix  then. 


where  X(X)  denotes  an  eigenvalue  of  the  matrix  X. 
If  A  is  square  symmetric,  then 

ii*ii*  ■  WA>  •  h*‘1ii2  ■ 

A1  so. 


(A-2) 

(A-3) 

(A-4) 

(A-5) 

(A-6) 

(A-7) 


1 


2liAii2 
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and  if  A  is  M  x  N  and  B  is  N  x  L,  then 

l|AB||F  ll!A||2||B|!F  (A-8) 

c)  For  the  sequence  of  symmetric  matrices 

Ar  =  {a(i  ,j)}  ,  (i,j)  =  1,2 . r  for  r  =  1,2,. ...N 

let  A^(Ar),  k  =  l,2,...,r  denote  the  eigenvalue  of  Af  where 

Xl(Ar)  >  X2^V  -  •••  -  MV 

Then  XkMAiM  -  W  -  MAi+P  (A-9) 

The  significance  of  this  result  [12]  is  that  the  smallest  eigenvalue 
of  a  matrix  sequence  is  non-increasing  with  size,  and  is  useful  in 
establishing  upper  bounds. 

Proof  of  Theorem  1; 

Let  S(z)  be  analytic  in  Zl  =  {y  <  |zj  <  1/y,  0  <  y  <  1}.  The  PA  condition 

on  S(z)  yields  i)  {r(£)>  forms  a  positive  definite,  exponentially  bounded 

sequence,  i.e.  |r(£)|  <  By^,  V£,  ii)  S_1(z)  is  PA  in  some 

Z?  =  (a  <  | z |  <  1/a  ,  0  <  a  <  1}.  Therefore,  if  S_1(z)  =  ?  r“(£)z-Jl  , 

£=-<*> 

z  £  Z2  then  |r  (£)|  =  Aa^  ,  V£.  From  (27),  for  the  infinite  non-causal 
MVR  we  have 

s_1(z)  =  -k1  -  ?  a(£)z-*]  (A- 10) 

6  £=-« 
tyO 

-  2 

which  means  r  (0)  =  1_  ,  and  a(£)  =  -B  r  (£),  V£.  Since  r~(i)  is 

B2 

exponentially  bounded,  this  means  ja(£)j  <  AB2a^  ,  ¥£  and,  the  limit  PEF 
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A(z)  =  1  -  l  a(i)z'i/  is  PA  in  1^.  Now,  subtracting  the  realization  equation 

£=-<= o 

WO 

(32)  for  the  p^  order  model  from  (26)  for  |n|<p,  using  the  definitions  in 

(33) ,  and  defining  a  vector  v  with  elements  v  (n)  =  l  a(«.)r(n-t) ,  |n|sp, 

P  M>P 

we  get 


V Vil  "  <62-S?,ip  +  ip 

where  a  -  [-a(-p)  -  a(-p+l)  -  -a(-l)  1  -a(l)  -  a(p)]T  . 

Now,  using  (33a)  and  defining 

a'  a 

Sp 

we  get  from  the  above  equation  that 


(4-11) 


^  “4  =  V  *p 


Taking  |)*ll2  norms  on  both  sides  of  the  above  equation  and  using  (A-7)  we  get 
l|a'-a||2  <  ||  R;l|l2l|Vpll2  • 


Using  the  bounds  for  (r(£)}  and  {a(£)}  and  evaluating  the  norm  of  vp,  we  obtain 


la^-allz  <  llR^llz 


2AB6^ay  /l-V 
1  “Y  /  1- 


4p+2 

T~ 


or 


Denoting  the  term  in  square  brackets  by  C  and  noting  that  |x(i)|  <  || J£.!l 2  ^or 
any  arbitrary  x,  the  above  inequality  yields  la^k)  -a(k)|  <  for  |k|<p. 

(A-8)  and  the  fact  that  WV  i  WW 1 . -  -  ■  1l’fCS(“)1  * c'  • 

[see  (A-9)]  we  obtain  |a'(k)  -a(k)|  <  Co11  ,  C"  4  C/C'.  Since  op  <  aj  o|ki 
for  |k|  <  p  and  aj  *  So  ,  we  finally  obtain 

|a^(k)-a(k)|  <  C"aJo|kl  ,  (k|  <  p  (A-12) 


Using 
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With  the  above  exponential  bound  in  (A-12),  it  becomes  a  simple  matter  to 
establish  uniform  convergence  of  the  sequence  of  PEFs  {Ap(z) }  to  A(z)  [16]. 

To  see  this,  we  write  the  expression  for  | A^(z)-A(z) | ,  use  (A-12)  and  the 
bound  for  |aU)|  to  yield 

|  A' (z)-A(z)  |  <  C"a?  £  a  I  k  I  |  z  rk  +  AB2[  ?  (a|z|)k+  ?  (ct|z_1|  )k] 

p  xk=-p  1  k=p+l  k=p+l 

al  6 

In  the  neighborhood  =  {■£=■  <  jz|  <  ,  a^<6<l}  the  above  bound  can  be 

proven  to  be 

|A£(z)-A(z)|  <  C  eP  (A-13) 


where  C>0  and  0<e<l  are  constants  depending  on  6,  etc.  Substituting 
for  a*  in  terms  of  a  [see  (A-ll)],  it  is  easy  to  prove  that 

C02eP  b2-b2 

lAp(z)-A<z)|  <-| - 4ep 


(A-14) 


where  C'  =  inf[|S(z)|]  in  some  neighborhood  of  |z|  =  1. 

Hence,  { Ap(z ) }  converge  uniformly  to  A(z)  in  some  neighborhood  of  |z|  =  1. 
It  has  been  proven  that  {Bp}  is  a  monotone  non-increasing  sequence.  Since  S(z) 

p 

is  PA,  B  in  (30)  exists  and  the  sequence  converges  to  this  limit. 

Stability:  Since  S(w)  >  0  and  analytic,  A(co)  is  analytic  and  has  no  zeros 
on  the  unit  circle.  From  (A-14)  we  obtain  Ap(co)  >  A(w)  -  0p.  Hence,  a  model 
order  pQ  can  be  chosen  so  that  for  all  p  >  pQ,  inf[A(co)]  -  0p  >  0,  guaranteeing 
stability  of  all  non-causal  models  of  order  greater  than  pQ. 


Correlation  Match: 

For  p  >  p0,  |S(u>)-Sp(u))| 


< 


K'e 


P 


In  writing  in  the  above  we  have  used  (A-ll),  (A-12),  the  least  upper  bound  for 
S(u)  and  the  fact  that  Sp(aj)  can  be  uniformly  bounded  from  above  for  p>pQ.  If 
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(rp(k)}  denote  the  covariances  realized  by  the  p  order  model,  then 

I  r(k)  -  rp(k)  |  <  ±  |S(u>)  -  Sp{u»)|dai  <  K'eP  . 

Thus,  the  correlation  match  gets  better  (exponentially  so)  with  increasing  model 
order. 

Proof  of  Theorem  2: 

Subtracting  the  realization  equation  (42),  for  the  infinite  order  MVR  from 
(48),  the  realization  equation  for  the  (p,q)  order  model,  for  (k,x.)eW3  and  using 
the  definitions  in  (49),  we  obtain 

oo  p  Q 

R  [a  -  a]  =  [6  -  8„  „]1  _  +  E  I  a(m,n)r(k-m,£-n)  +  E  E  a(m,n)r(k-m,Jl-n) 

P.q  “P.9  P»q  "H'.q  m=_p  |n|>q  n=-o  lml>D 


n=-q  I m | >p 


(k,Jl)eW- 


where  a  is  the  vector  of  elements  (a(m,n)},  (m,n)£W3  arranged  corresponding  to 

the  elements  of  a  The  summations  are  interpreted  as  vectors  with  elements 
P 

indexed  in  (k,£).  Defining 


a!  -  6/  a  „ 

~p  ,q  /62  “P  ^ 

p.q 


(A-15) 


we  can  write 


al  -  a  =  R"1  [  E  E  (•)  +  E  E  (•)]  .  (k,fc)eVL 

p,q  m=-p  1 n 1 >q  n=-q  |m|>p 

where  (•)  denotes  the  quantities  to  be  summed,  i.e.  {a(m,n)r(k-m,£-n)}. 

Now  taking  Frobenius  ( ||  •  ||  p)  norms  on  both  sides  of  the  above  equation, 
using  (A-8),  the  bounds  in  (4)  for  r(k,«.)  and  a(k,t),  and  evaluating  the 
resulting  summations,  we  find  that  for  some  positive  constants  Cj.Cg.C^. 

Il5p,q_— Hf  -  llRp!qll2^Clotl+C2°^+C3al0^^ 
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Using  (fl-6) ,  ||R^qil  2  -  lAmj „(%,„)•  WW  1s  TO"otonici,’ly 

non-increasing  with  increasing  (p,q)  [see  (A-9)]  with  limiting  value  S  =  inf  Sfwj.Wj)- 

the  above  bound  may  be  written  as 

Z  [a*  (m,n)  -  a(m,n)]2  £  T(C,aP+C2<X2+^3aia2^ 

(m,n)eW3  p,q 

2 

Hence,  lim  Z  [a*  (m,n)-a(m,n)3  =  0 

p,q-*»  rn.neW^  P*M 

This  implies  that 

lim  I  |a*  (m,n)  -  a(m,n)|  =  0  (A-16) 

p,q-^°  rn.neW^  p,q 

i.e.  the  sequence  {a^  ^(m.n)  -  a(m,n)},  indexed  in  (p,q),  with  (m,n)eW3  is 
absolutely  summable  on  the  unit  circles,  |Zj|  =  |z2|  =  1  and  converges  to  zero 

as  (p,q)  -*■  “ 

Now 


k 


lAi,q(zrZ2)'A(zl’Z2)|  =  (k  j)eW  [aP*q(k,J')"a(k’0]Zl  22 


z  a(k,t)z7kzlx' 
(k,t)^W3 


Using  the  bound  for  a(k,£)  and  evaluating  the  second  summation,  we  get 
for  some  positive  constants  Kj,K2> 

l'W2l-22>-*<2l-22»  i  2  lap,q(k't,-a(k',')l  *  Kl“l  *  V»2  1 

’  '  3  lijl  ■  U2I  •  1 


Letting  e(p,q)  ■  _  ?  la^q(k,i)-a(k,t)  |  + 


k.teWj 


we  have 
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lAp,q(Zl*Z2)'MZl,Z2)!  <  e(p.q)  »  ! Zjl  =  lz2l  =  1  (A‘17) 

Since  lim  e(p,q)  =0  it  follows  that  lim  A'  (z-.zj  =  Mzj.zJ  uniformly 

p,q-«o  (p.q)-*”  P,q 

on  |zx |  =  |z2l  =  1. 

P  o 

Now,  {$  }  is  non-increasing  with  (p,q)  with  lower  limit  8  given  by  (46), 

p  *q 

since  by  assumption  S‘1(w1 ,uj2)  exists.  Thus,  similar  to  the  1-D  non-causal  case, 

I Ap ,q(zi ,z2)-a(zi »z2) I  <  ^'(p.q)  *  !zil  =  lz2l  =  1 

This  establishes  the  uniform  convergence  of  the  PEFs  to  the  limit  PEF  on 
the  unit  circles. 

b)  Stability:  A(z1,z2)  is  PA  on  the  unit  circles.  Let  a  =  inf  A(o^,uj2)  >  0. 

Since  |A  .w^-A^  ,u>2)  |  <  e'(p,q),  we  can  find  some  (po,qQ)  <  00  such 
that  for  (p,q)  >  (pQ,q0),  e’(P.q)  <  a  implying  that  A^.aig)  >  a  -  e'(p,q)  >  0  , 

(p.q)  >  (p0*%) 

Hence,  the  non-causal  models  are  guaranteed  to  be  stable  for 

(P.q)  >  (P0»90)* 


c)  Correlation  Match: 


For  (p.q)  >  (P0»q0)» 

|  S(d)^ »w2) -Sp  I 


The  above  bound  is  obtained  similar  to  the  1-D  non-causal  case.  Hence, 
it  follows  that 


|  r(k,£.)-rp,q(k,£,)  |  <  k  e(p,q)  ,  (p.q)  >  (?„.%)  • 
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Proof  of  Theorem  3: 

Proof:  Since  Sfz^.Zg)  is  PA,  SCzj.Zg)  =  log  Sjz^Zg)  is  also  analytic 
in  a  neighborhood  of  the  unit  circles,  say,  Zj  =  {a^lzj^l/aj,  o^^Hl/c^; 
0<a1»a2<l}.  A  unique  Laurent  expansion  for  S  in  can  be  written  as 

S(z1  ~  E  c(m,n)z^  Z2  , 
m,n 

The  above  series  can  be  decomposed  as  S  =  H+  +  H~  +  S^.  [10]  where 

H+  =  I  c(m,0)zrm  +  E  ?  c(m,n)zimz2n 
m=-°°  n=l  m=-» 

H"  =  f  c(m,0)zjm  +  ?  E1  c(m,n)zjmz2n 
m=-°°  m=-°°  n=-°° 

S  =  -  ?  c(m,Q)z7m 
m*-<» 

and  are  analytic  in  Z 2  *  {at1< | J <1/0*^ » | z2 } >ot2>  >  lj  ~  (a^lzj^l/aj,  | 1  <ct2> » 
and  Z^  =  {a1<|z1!<l/a1},  respectively.  The  positive  monotone  property  of  the 
exponential  function  implies  that  e^  ,  e±^  ,  and  e  c  are  also  analytic  in 
Z2,  Z3,  and  Z4,  respectively. 

Hence,  we  may  write 


A  -H 

Defining  A '(zj.Zg)  -  e  ,  we  have 

StZj.Zg)  =  a^ZjJ/A'fZi.ZgjA'tz^.Zg1) 

S 

where  a^Zj)  *  e  e  =  A'Uj,*),  is  a  1-D  SDF. 
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The  arguments  above  show  that  A‘(Zj,z2)  is  semi-minimum  phase.  With 


A(z1,z2)  -  1  -  E  „  a(m,n)z'"'z'"  =  BtA'(z,,z2) 
m ,  n  c  S  2 


-m  -n  _  n2. 


we  get 


ran(z.) 

S(z,,z2)  = - M - r- 

A(z1,z2)A(z1i,z21) 


With  the  given  cepstral  decomposition,  and  the  requirement  that  a(0,0)=l, 
it  follows  that  the  decomposition  is  unique. 


Proof  of  Theorem  4: 

The  proof  of  this  theorem  is  similar  to  that  of  Marzetta's  [7,8]  for 
An(z1,z2)  from  which  the  contention  for  An(zj>z2)  follows.  The  convergence  of 
the  SDFs  can  be  shown  similar  to  Theorem  1  where  all  the  quantities  are  now 
functions  of  z^.  The  detailed  proof  may  be  found  in  [16]. 


Proof  of  Theorem  5: 

a)  With  the  PA  assumption  on  the  SDF,  S(z1>z2)  and  following  the  method 
of  proof  in  Theorem  2,  it  is  easy  to  prove  that 

E  [a‘  „(m,n)-a(m,n)]2  <  C.o?  +  C«a5  +  C0a?a5  ,  0<a, ,a,<l 
(m,n)eW2  p’q  1122312  12 

where 


ap,q(n,,n)  “  ~f —  ap,q(m,n)  »  (m,n)e  W£ 

SP,q 


(A-18) 


Hence , 


lim  E 
p,q-**>  (m,n)eW2 


an  a(m*n)  "  a(m*n) 
F 


(A-19) 


Inlying  that  the  sequence  (a'  „(m,n)  -  a(m,n)},  indexed  in  (p,q) 

P 

with  (m,n)  e  Is  absolutely  summable  on  the  unit  circles,  and 
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converges  to  zero  as  (p,q)-w°. 

Now,  for  Z1  =  {|Zj|  =1,  |z2|  _>  1  , 


m, _  , -n 


iAno(zrz2)'A(zl,z2,l  -  E  |a'  (m,n)-a(m,n)||zir  |z2 

p,q  1  £  1  *  (m,n)eW  p,q  i 


+  Z  )  a  (m  ,n )  |  |z,  |"m|z9|"n  <_  E  |a'  (m,n)-a(m,n)| 


(m,n)£W 


(m,n)eW, 


+  Z  |a(m,n) j  =  e(p,q). 
(m,n)£W0 


(A-20) 


The  PA  assumption  for  S(z1>z2)  implies  that  {a(m,n)}  are  exponentially 
bounded.  This  and  (A-18)  together  imply  that  lim  c(p,q)  -►  0 

p,q->«, 

Hence,  lim  A'  (z,  ,z5)  -*•  A(z.  ,z~)  uniformly  on  (z,  ,z9)  e  Z. . 

p  ,q-K»  p  ,q  1  c  1  c  L  c  1 

Similar  to  non-causa  1  models,  it  can  be  established  that  (6  }  is  a 

P  »M 

2 

nonincreasing  sequence  that  converges  to  a  positive  lower  limit,  6  . 

p 

We  now  give  an  expression  for  8  ,  which  is  also  given  in  [10]. 

Defining  d(z1)  -  g2ag*(Zj),  and  from  Theorems  3  and  4,  it  follows 
that  d(z^)  is  PA  in  a  neighborhood  of  |zj|  =  1,  so  that  d’^Zj)  is 
likewise. 

Now,  62  =  Ay  ffi  Se(co1,a)2)dw1dw2  =  ^  /^e2a0(a)1)dw1 
4tt 


Thus, 


e2  =  ife  /J  d"1^)*^  >  o 


Using  (A18-A20),  we  obtain 


lAp.q(zl-z2)-fl(2l-22)l  <  <A'21> 


,2  4 


zrz2 e  zi 


where  A(nflx  A  sup| A(Zj ,z2) |  ,  Zj,z2  e  Zj. 
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This  proves  the  uniform  convergence  of  the  sequence  of  the  semi-causal  PEFs 
A?  t0  ana1ytic*  semi-causal  limit  PEF  A^^)  in  Zl* 

b)  To  prove  stability,  we  note  from  (A-21)  that 

I rp ,q”r I  <  e'(P»q) 
and  ! ip tq-i I  <  e'(p,q) 

where  r  ,  r  and  i  ,i  represent  the  real  and  imaginary  parts  of 

r  * H  r 

Ap  q(zi»z2)  and  A(zj,z2)  for  any  Zj,z2  e  Z^.  Since  r  and  i  are  not 
simultaneously  zero,  we  can  find  some  (P0»cl0)  <  00  such  that  for 
(p,q)>(p0,qQ) ,  rp  and  i  are  sufficiently  close  to  r  and  i, 
respectively.  This  implies  that  the  models  will  be  stable  for 

(p,q)>(p0»q0) - 

c)  We  now  prove  uniform  convergence  of  the  spectra  realized  by  the  models 
for  (p,q)  >  (p0»<10)  to  the  given  SDF  on  the  unit  circles.  This  in 
turn  implies  increasingly  accurate  correlation  match.  Since  for 
(P.q)  >  (p_,q_).  A  (z,,z9)  has  no  zeroes  in  Z, ,  it  is  semi-minimum 
phase,  and  it  is  easy  to  establish  the  uniform  convergence  of 

i/Ap ,q(zi >z2^  t0  1/A(z1 ,z2) .  Also,  since  a0Pjq(zi)  =  Ap,q^zi’“^»  n 

follows  that  aQp  q(z^)  converges  uniformly  to  3q(zj)  on  |zj|  =  1. 

2  ? 

Since  8  n  +  8  monotonically,  and  since  the  PEF  sequence  can  be 
P  »M 

uniformly  bounded,  it  follows  that 


■i  _-i> 


*  V2!* 


.-i  -i. 


Ap,q^zl,z2^Ap,q^2l  ,z2  ^  AUj.ZgjAfZj  ,z2  ) 
uniformly,  i.e. 

(Sp^qtZj.Zg)  -  S(Zj ,Zg)  |  <  c"(p,q)  I z j I  s 


l*jl  -  lz2l  •  1 


|z2l  *  1 


In  proving  the  uniform  convergence  of  the  SDFs  from  that  of  the  PEF 


sequence,  we  obtain  a  bound  e''(p,q)  which  may  be  weaker  than  e'(p,q). 
However,  lim  e"(p,q)  =  0. 

(p»q)-*° 

If  (r  _ ( k ,R.) }  are  the  covariances  realized  by  the  (p,q)  order  model, 
P 

then. 


1  77 

!rp,q(k»^)-r(k‘£)  1  -J2.  //'Sp,q(u)l’u>2)  “  S(u1,u2)|du1du2  =  e"(p,q) 


This  proves  that  correlation  match  gets  increasingly  better  with 
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Figure  1:  Prediction  regions  and  typical  prediction  windows  for  causal,  semi- 
causal,  and  non-causal  geometries. 

Figure  2:  An  example  of  semi-causal  filter  geometry  obtained  by  the  rationali¬ 
zation  procedure. 

a)  Reflection  coefficient  geometry 

b)  Geometry  for  an(m) 

c)  Prediction  geometry  for  the  semi-causal  model 

Figure  3:  A  method  of  obtaining  finite  order  semi-causal  models  via  Levinson's 
recursions. 

PI  2 

Figure  4:  Covariances  corresponding  to  r(k,£)  =  .9/k  +£  on  the  quarter  plane. 

Figure  5:  PEFs  and  covariance  mismatches  for  (p,q)  =  (1,1)  and  (2,1)  non-causal 
models. 

a)  PEF  coefficients  corresponding  to  (p,q)  =  (1,1)  non-causal  model 

b)  PEF  coefficients  corresponding  to  (p,q)  =  (1,2)  non-causal  model 

c)  (r(k,£)-rljl(k,£)} 

d)  {r(k,«,)-r1  2(k,X,)} 

Figure  6:  Infinite  order  semi -causal  PEF  obtained  by  Wiener-Doob  factorization. 

Figure  7:  Rationalized  semi-causal  model  corresponding  to  q=2,  {pj (k) }  trun¬ 
cated  to  5  terms,  {p2(k)}  truncated  to  3  terms,  and  p=2. 

a)  Coefficients  of  semi -causal  PEF 

b)  Covariance  mismatch,  (r(k,£)-r(k,Jt)} 

Figure  8:  Semi-causal  model  obtained  using  q=l,  truncating  { p, ( k ) )  to  15  terms, 
and  p=2.  1 

a)  Semi-causal  PEF  coefficients 

b)  Covariance  mismatch,  {r(k,£)-r(k,t)} 

Figure  9:  Semi-causal  models  obtained  by  solving  finite  order  linear  equations, 
a)  PEF  coefficients  of  (p,q)  *  (1,1)  semi-causal  model 


b)  PEF  coefficients  of  (p,q)  *  (2,2)  semi-causal  model 
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c)  PEF  coefficients  of  (p,q)  =  (3,3)  semi-causal  model 
Figure  10:  Covariance  mismatches  of  the  semi-causal  models  in  Figure  9. 

a)  {r(k,2)  -  r^U.O) 

b)  (r(k,£)  -  r2>2(k,£)} 

c)  {r(k,£)  -  r3j3(M)} 

Figure  11:  Infinite  order  causal  PEF  obtained  by  Wiener-Doob  Factorization 

Figure  12:  Rationalized  causal  model  corresponding  to  q=2,  (p^k)} 

truncated  to  5  terms,  { p2 ( k ) }  truncated  to  3  terms,  and  p=2. 

a)  Coefficients  of  causal  PEF 

b)  Covariance  mismatch,  (r(k,£)  -  r(k,£)> 

Figure  13:  Rationalized  causal  model  obtained  using  q=l,  (p^k)}  truncated 
to  15  terms,  and  p=5. 

a)  Coefficients  of  causal  PEF 

b)  Covariance  mismatch,  (r(k,il)  -  r(k,t)} 

Figure  14:  Causal  models  obtained  by  solving  finite  order  linear  equations. 

a)  PEF  coefficients  of  (p,q)  *  (1,1)  causal  model 

b)  PEF  coefficients  of  (p.q)  =  (2,2)  causal  model 

c)  PEF  coefficients  of  (p,q)  -  (3,3)  causal  model 

Figure  15:  Covariance  mismatches  for  the  causal  models  in  Figure  14. 

a)  (r(k,£)  -  rj j(k,£)} 

b)  (r(k,£)  -  r2>2(k,£)} 

c)  {r(k,H)  -  r3>3(k»^)) 


